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MULTI-PLAYER STOPPING GAMES IN CONTINUOUS TIME 


ZHOU ZHOU 


Abstract. We consider multi-player stopping games in continuous time. Unlike Dynkin games, in 
our games the payoff of each player is revealed after all the players stop. Moreover, each player can 
adjust her own stopping strategy by observing other players’ behaviors. Assuming the continuity 
of the payoff functions in time, we show that there always exists an e-Nash equilibrium in pure 
stopping strategies for any e > 0. 


1. Introduction 

Since [3], Dynkin game has attracted a lot of research. We refer to [3H7l[9l-[T8] and the references 
therein. In a Dynkin game, each player chooses a stopping strategy, and the payoffs of the players 
are revealed as long as one player stops. In other words, the game ends at the minimum of the 
stopping strategies. With some assumptions on the relationship between the payoff processes, it 
is proved in e.g., mm that a two-player non-zero-sum Dynkin game in continuous time admits 
a Nash equilibrium in pure stopping times. In general, without such assumptions, the two-player 
Dynkin game only has a Nash equilibrium in randomized strategies, see e.g., [12]. It is known that 
when there are more than two players, the Dynkin game in continuous time may not have any Nash 
Equilibrium in randomized strategies even if the payoff processes are constant (see e.g., |13]). 

As a classical model of stopping games, Dynkin game has many applications in economics and 
finance. However, it has two major limitations in many situations. First of all, in practice, it is 
more often that, even if a player has made the decision first, her payoff can still be affected by 
other players’ decisions later on. In other words, it is more reasonable that the games end at the 
maximum of the stopping strategies. Second, a wise player would adjust her strategy after she 
observes other players’ actions, and Dynkin games cannot incorporate this “game” feature. 

Recently [Tl[2lll9ll20j begin to consider the stopping games with these two features. In particular, 
assuming that the payoff functions are continuous in time, m shows that a two-player non-zero-sum 
stopping game always admits an e-Nash equilibrium in pure strategies for any e > 0. 

In this paper, we extend the result in m to the multi-player case. To be more specific, given a 
filtered probability space (D, J”, (-7^)t6[o,oo] j 1®)) we consider the stopping game in continuous time 

u*(p\... ,p^) = E[[/*(p\... ,p^)], i = ,N, 

where the player i chooses p* to maximize the payoff rtb Here W{ti,... , t^) is Ji^v...viAr-™easurable 
instead of Ji^A...AiAf-™easurable as is assumed in Dynkin games. That is, the game ends at the 
maximum of players’ stopping. Moreover, here p® is not a stopping times. It is a strategy that 
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can be adjusted according to other players’ actions. By assuming is uniformly continuous in 
{ti, ... we show that the game always admits an e-Nash equilibrium in pure strategies for any 
e > 0. 

We prove the result by an induction on the number of the players. That is, we construct an 
e-Nash equilibrium of the Wplayer game from the e-Nash equilibriums of {N — l)-player games as 
well as e-saddle points of some zero-sum games. To reduce the burden of the notation, we only 
focus on the three-player case {N = 3), and the proof still works accordingly for the case with more 
players. 

Our game has a wide range of applications, e.g., when companies choose times to take actions, 
and e.g., when investors who both short and long American options try to maximize their utilities. 

The paper is organized as follows. In the next section, we provide the setup and the main result. 
In Section 3, we provide some auxiliary results. In section 4, we use these auxiliary results to 
construct an e-equilibrium for the original game. 

2. Setup and the main result 

In this section, we will provide the general setup and the main result. Theorem 12.61 is the main 
result of this paper. 

Let (n, (•^t)tg[o,oo])IP) be a filtered probability space, where T” = Too and (.Tt)jg[o,oo] satisfies 

the usual conditions. To avoid the technical difficulties stemming from the verification of path 
regularities of some related processes, we assume that is at most countable, and P is supported 
on Q,. Let T be the set of stopping times. For p G T, denote Tp (resp. Tp+) be the set of stopping 

times that is no less (resp. strictly greater) than p on {p < oo}. 

Definition 2.1. Let N G and I be the set of all the subsets of {!,... ,N — 1}. A 2^~^- 

tuple p = {pa)aei is said to be a stopping strategy (of order N), if p := p^ G T, and for any 

/ = (ii,. .. ,in) C {1,... , - 1} with h < ... < in, 

Pii,...,in ■ [OjOc]"" X n !-)■ [0,oo] is 0([O, oo]"") 0 T-measurable, 

and 

Pil,... G 7(iiV...Vt„)+- 

Denote T'^ as the set of stopping strategies of order N. For a G F, let 

:= {p = : p>a}. 

The interpretation of p € is as follows. Suppose there are N players, and each of them needs 
to choose a time to make a decision (stop). Let p be player N's stopping strategy. At the beginning, 
player N chooses an initial stopping time p. If no other players stop before p, then player N stops 
at time p. Otherwise, player N observes someone stops before p. Say, it is player 1 who stops 
first at time t < p. Then player N observes player I’s action, and immediate switches to strategy 
pi{t). In general, pq,...,i„(ti,... ,tn,-) represents the strategy that player N uses, if she has not 
stopped by time tiM .. .M tn, and she observes that players ii,... ,in have stopped at time ti,... ,tn 
respectively. 
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Let p® € T-^, i = 1,... ,N, which presents the stopping strategy for player i in an A^-player game. 
Denote p®[p~*] as the actual time when player i stops under the other players’ stopping strategies 
p“® := (p^,... ,p®“^,p®'''^,... ,p^). Due to the complexity of the notation, we will not explicitly 
write out the expression of p®[p“®]. Instead, we give two examples when N = 2 and N = 3. 

Example 2.2. Let p = (p, pi),T = (r, Ti) G T^. Then 

p\t\ (^) ^{p>t} ) 

where pi(t) := Pi(t(-), •)• 

Example 2.3. Let p = {p, P2, P3, P23 ),t = (d D, T 3 , ns), £T = (cr, ui, ( 72 , 0 - 12 ) € Then 

pf{p<TAa-} P23('^)'^)l{p>T=cr} 

P3 ('^) f {p,r>(T}n{p 3 ((t)<T 3 (ct)} T P 23 (LS (*^))'^) f {p,r>(T}n{p 3 ((t)>T 3 ((t)} 

“ 1 “ P 2 ('r)l{pja->r}n{p 2 (T)<( 72 (r)} “ 1 “ P23 ('T, (72 (t)) ,j^.r}n{p 2 (t)>(T 2 (r)} ) 

where p 23 {T,(r) := p23(r(-), cr(-), •)• 

For i = 1,... ,N, let W : [0, oo]^ x D i-)> R such that ... , tjy) is L't^v...vtjv"™ 68 ’Surable. We 

make the following standing assumption throughout this paper. 

Assumption 2.4. There exists a bounded nondecreasing function p : R+ R+ satisfying lim, 5 ^o vi^) 
r/(0) = 0, such that for i = 1, ... ,N, and any (fi, ... , tpf), {t[,... , t^) € [0, oo]^, 

|17®(ti,... , tj^) — U^it'i ... , t^)| < pdti — I + ... + \tN “ ^tvD- 


Now for any 9 T, consider the A-player stopping game in continuous time 

u\p\... ,p^):=Ee[U\p^[p-%... ,p^\p-^])], p\...,p^eT^, i = l,...,N, (2.1) 

where IEp[-] := E[-|Ap]. Here player i chooses the stopping strategy p® to maximize her own utility 
t/®. Recall the definition of an e-Nash equilibrium. 


Definition 2.5. For e > 0, the N-tuple {p^,... ,p^) € is said to be an e-Nash equilibrium 


(in pure stopping strategies) for the game (12. Ih , if for any p ^, 


,P^€T^ 


u\p\. 


■ p ,p ,p , 


,p 


N\ 


<u®(p\...p® \p®,p®+\ 


,P^) + e, 


i = l,. 


,A. 


Below is the main result of this paper. 


Theorem 2.6. Under Assumption 1^-41 there are exists an e-Nash equilibrium for the game (12.11) 
for any e > 0. 


Remark 2.7. Our game can also be adapted to the case when each player has multiple stopping. 
For example, suppose each player has double stopping. Then we can treat player i as new players 
2i — 1 and 2i, and let := t/^® := 1/®. Of course, by doing so, a Nash equilibrium in the new 

game may not be optimal for each player in the old game. (Recall that it is possible that a Nash 
equilibrium {x*,y*) for f^{x,y) = f{x,y), i = 1,2 may not be optimal for siip,^yf{x,y).) 
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We will prove Theorem 12.61 by an induction on the number of players. We will construct an 
e-Nash equilibrium of the Wplayer game from e-Nash equilibriums of {N — l)-player games, as well 
as some e-saddle points of some related zero-sum games. By m Theorem 2.4], Theorem 12.61 holds 
when N = 2. (Although in [19] the game starts from time t = 0, but the proof there still works if 
the game starts at any stopping time.) To reduce the burden of the notation, we will only prove 
the result when N = 3. It can be seen later on that our proof also works for general N, as long as 
we assume that Theorem 12.61 holds for — 1. 

We will first provide some auxiliary results in the next section. 


3. Some auxiliary results 

In this section, we provide some auxiliary results in preparation for the proof of Theorem 12.61 
when N = 3. Some of the proofs may admit simpler solutions, yet we demonstrate them in such 
ways in order to let the proofs also work for N > 3. It is worth noting that as 12 is at most 
countable, those results that hold w.r.t. any t G [0, oo] also hold w.r.t. any 6 

Lemma 3.1. For i = 1,2, let G* : [0,oo]^ x 12 i— >■ M such that G^{r,s,t) is Frysyt-fneasurable and 
satisfies Assumption \2.4\ Then for any e > 0, there exists h > 0, such that for any t G [0, oo), there 
exists (p,t) G (TI)^ sueh that for any 6 G [0, /i], (p,f) is an e-Nash equilibrium for the game 

Et-s[Gfip[T],T\p],t-6)], p,T£T‘f_s, i = l,2. 

Proof. For any e > 0, let /i > 0 such that ry(/i) < e. Now for any t G [0, oo), by [IHl Theorem 2.4], 
there exists an e-Nash equilibrium {p* = {p*,pl),T* = (T*,rf)) G {Ttfi for the game 

Et [Gfip[T],T[p]F)] , P,T gT?, i = l,2. 

Let p := (p*,/5i) and r := (r*,fi), where 





if s <t p*, 


rfit), 

if s < t / r* 

t, 

if s <t = p*, 

and Ti(s) = < 

t, 

if s < 2 = T* 

Plis), 

\ 

if s >t, 


Al 

_✓ 


It can be shown that p,f ^ T^, and for any p,T £ T^, 

p[t] = p*[t], t[p] = t[p*], f\p] = T*[p], p[f] = p[t*]. 
Then (p,t) is a 3e-Nash equilibrium for the game 


Et[Gfip[T],T\p\F - 5)] , P,T 


i = 1,2. 




for any 5 G [0, /i]. Indeed, for any p G , 
Kt [G^{p[f],f[p],t - S)] 


Now take p = (p, pi) G We have that 

[G^{p[riT\p\,t - 5)l{p<t<r*}] = 

< 


< 


and 

Et-s [G^{p[f],f[p],t- = 

< 


< 


and 

Et-5[G^{p[f],f[p],t-6)l{p>t}] = 

< 


where p := (t, pi(s) = s + 1) G T^. Therefore, 
Et_5 [G\p[f],f[p],t - 5)] : 
Similarly, we can show that for any r G 

[G‘^{p[T],T[p],t - 5)] : 


: Et[U\p[f],f[p],t)] +e 
■ Et [GHp[r*],r*[p],t)] +e 
: Et[G\p*[T*],T*\p*],t)]+2e 
: Et[GHp[r*],f[p*],t)]+2e 
: Et[G\p[f],T[p],t)] +2e 
: Et[G\p[f],f[p],t-S)]+3e. 

Ei_, [Ei [G\p,T^{t),t- ,5)] l{p<t<.*}] 

Ei-(5 [Ej (i),i — (5)] l{p<t<T*}] +e 

Et_5 [Et [G^(p[f],f[p],t-5)] l{p<i<r*}] +e 
Ef-^ [Ef [G^(p[f],f[p],f — 5)] l{p<i<r*}] +4e 
Ei-5 [G^{p[f],f\p],t - 5)l{p^t<T-}] +4e, 

Ei_, [Ei [G\p,t,t-6)] l{p<t=T*}] 

Et_5 [Et [G^{t,t,t - 6)] l[p^t=T*}] +e 
Et_5 [Et [G\p[f],Tlp],t - 6)] l{p^t=T*}] +e 

Et-s [Et [G^(p[f],f[p],f-5)] l{p<t=r*}] +4e 

Et_5 [G^{p[f],f[p],t - 5)l{p^t=T-}] +4e, 

Et_5 [Et [G^{p[f],f[p],t - 5)] l{p>t}] 

Et-5 [Et [G^(p[T],f[p], f — 5)] l{p>t}] + 3e 
Et_5 [G^{p[f],T[p],t - 5)l{p>t}] + 3e, 

Et -5 [G\p[f],T[p],t - S)] +lle. 

Et _5 [G'^{p[f],T[p],t - 5)] +lle. 


Lemma 3.2. Let G : [0,oo]^ x fl i-)> R such that G{r,s,t) is Fr\/s\/t-fneasumble. Then 


inf Et[G(p[r],r[p],t)] = inf Et[G(p,T,f)] . 
.reTt P,'r&Tt 
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Proof. Fix t € [0, oo). For any p,T ^ Tf, p[t],t[p] G 7t, and thus 


This implies that 


p,rGT2 

Conversely, for any p,T let 


^t[G{p[T],T[p],t)] > inf IEt[G(p,T,t)] 


inf Et[G{p[T],T\p],t)] > inf Et[G{p,T,t)] . 

p,r£Tt 


p, if s < p, r, if s < r, 

pi(s) = < and Ti(s)=< 

I oo, otherwise, I oo, otherwise, 

and p = (p, pi),r = (r, ri). Then p,f G T^, and p[f] = p and T[p] = r. Therefore, 


This implies that 


inf Et [G(p[t] , r[p] , t)] < E* [G(p, r, t)] . 

p.-reT? 


inf Et[G(p[T],r[p],t)] < inf E* [G(p, r, t)] . 


p,rGT2 


p,rG7t 


(3.1) 


□ 


Lemma 3.3. Let G : [0,oo]^ x i->- M such that G{r,s,t) is Fr\/s\/t-fneasurable and satisfies 
Assumption 12-41 Then for any e > 0, there exists h > 0, such that for any t G [0, oo), there exists 
(p,f) G (Tj)^ such that for any 5 G (p,t) is an e-optimizer for 

gt-s ■= inf Et _5 [G(p[r],T[p],t - (5)] . (3.2) 

That is, for any 5 G [0, h], 

Et-s [G{p[T],T\p],t - 5)] <gt -5 + e. 


Proof For any e > 0, let /i > 0 such that p(/i) < e. Fix t G [0, oo). By Lemma [321 

gt = inf Et [G{p, T,t)] . (3.3) 

pn^Tt 

Let {p*,T*) G (7))^ be an e-optimizer for gt in (13.3p . Define p,f G as in (13.ip such that p[f] = p* 
and t[p] = r*. Now for any 6 G [0, h], we have that 

Et-5[U{p[f],f[p],t-6)]-gt-5 = Et-5[U{p*,T*,t-6)]-gt-5 


< 

< 

< 

< 

< 


E 4_5 [Et [U {p*,T*,t)]]- gt-s + e 


Et_ 


t-5 


inf Et [U{p,T,t)] 
p.reTt 


~ gt-S + 2e 


inf Et _5 [Et [U (p, r, t)]] - gts + 2e 

P,T&Tt 

inf Et _5 [17(p, r, t - 6)] - gts + 3e 

pn&Tt 

inf Et _5 [C/(p V t,r Vt,t - (5)] - pt -5 + 3e 

P,T€Tt-s 

inf Et _5 [U (p, T,t- J)] - gt-s + 5e 

pneTt-s 

5e. 




7 


This implies that (p,f) € (Tf)^ is a 5e-optimizer for gts in (13.2p for any <5 G [0, /i]. □ 

Lemma 3.4. Let H : [0, oo]^ x 0 i-G M such that H[s,t) is Fsyt-fn^o,surable and satisfies Assump¬ 
tion's^ Then for any e > 0, there exists h > 0, such that for any t G [0,oo), there exists p G Tt 
such that for any 5 G [0,/i], p is an e-optimizer for 

inf Et-s[H{p,t - 5)] . 

pGlt-S 

Proof. The proof is similar to that for Lemma 13.31 and thus we omit it here. □ 

Lemma 3.5. Let G : [0,oo]^ x hi t-G M such that G{r,s,t) is Fr\/s\/t-fneasurable and satisfies 
Assumption \2.4\ Then for t > 0, 

ft := sup inf Et [G{p[T],T[p],t)] = inf sup E* [G{p[T],T[p],t)] . (3.4) 

PGT? tGT? tGT? pgT? 

Moreover, the process {ft)t>o is right continuous. 

Proof. The equality in (13.411 follows from [T9l Proposition 3.3]. Now fix f G [0,oo) and let tn \ t 
with \tn — < 1/n-. For any e > 0, by Lemma [3Tl there exists h > 0 and {p,f) G such that 

for any 6 G [0, h], (p,f) G (T^)^ is an e-saddle point for the game /t_h, i-e., for any p,T G Tj_^, 

Et _5 [G{p[f],f[p],t- 5)] - e < Et _5 [G{p[f],T[p],t - <5)] < E^.^ [G{p[T],T[p],t - <5)] -h e. 

Then for n large enough such that — t\ < h, we have that 

\ft„ -ft\ < |/t„ [G{p[f],T[p],tn)]\ + |Et„ [G{p[f],T\p],tn)] -Et„ [G {p[f], t[p] , t)]\ 

+ |Etn [GW]F[p],t)] - Et [G{p[f],f[p],t)]\ + |Et [G{p[T],T[p],t)] - ft\ 

< € + p{l/n) + |Et„ [G{p[f],f[p],t)] - Et[G{p[f],T[p],t)]\+e. 

Since the process {Et+s [G'(p['^])7^[p])i)])s>o is a martingale w.r.t. the hltration (J't+ 5 )s>o satisfying 
the usual conditions, by [U Theorem 3.13, page 16], it is right continuous. Therefore, 

limsupl/t„ - ft\< 2e. 

n^oo 

By the arbitrariness of e, lim„_>.oo = ft, and thus the conclusion follows. □ 

Lemma 3.6. Let G : [0,oo]^ x hi i-G M such that G{r,s,t) is Fr\/s\/t-fneasurable and satisfies 
Assumption \2.4\ Then the process {gt)t>o defined by 

gt:= inf Et[G(p[r],T[p],t)] 

P.-reT? 

is right continuous. 

Proof. The proof is similar to that for Lemma 13.51 and thus we omit it here. □ 

Proposition 3.7. Let U : [0,oo]^ x i—>■ M such that U{r,s,t) is Frysyt-fneasurable and satisfies 

Assumption \2.4\ For t > 0, let 

Vt := sup inf Et[C/(p[T,o-],r[p,(T],(T[p,r])] 

peT? t,^tGT3 


(3.5) 
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and 


Then for any fi £ T, 


Vt ■■= inf snpKt[U{p[T,(T],T\p,(T],(T\p,T])] 


y^^ = Vf,= sup inf Ep [^pl{p<e} + ^el{p>0}] = inf sup Ep [Xpl^p^gy + l0l{p>e}] , (3.6) 

p€7^ p&% 


where for t >Q, 


and 


Xf.= inf ¥.t[U{t,T[(T],CT[T])] 


Yt := sup inf Et [[/(p[a],f,a[p])] A sup inf Ej [[/(p[t],t[p], t)] . 

J \pGT|^eT2 J 

Moreover, there exists an e-saddle point for (133]) with t replaced by p for any e > 0. 
Proof. Denote 

F/:= sup mf Et[U{p[a],t,a[p])] and F/:= sup inf Ej [C/(p[T],r[p],t)] 

pgt? <^eT2 pgTr2 tgt? 

For t > 0, let 0 = {t, Oi{s) = s + 1) € Tf. Then 

Xt< miEt[U{t,t,a[e])]<Y,\ 

<^eT2 

Similarly, we can show that Xt < Y^^. Hence, 


Xt < Yt. 


Moreover, by Lemmas 13.51 and iTGl {Xt)t>o and {Yt)t>o are right continuous. Then from the classical 
theory of Dynkin games, we have that for t > 0, 


Vt := sup inf E* [Xpl{p<0| + Fpljpx?}] 
peTt u 

= sup inf Et [Xpl{p<0} + FM{p>0}] 

peTt ^GTt 

and the process {Vt)t>o is right continuous. 

Now fix /r € T and let e > 0. Define p,9 £ Tp as 

p := inf{t > p : Vt < Xt + e} and 


= inf sup Et [Xpl{p<p} + FM{p>0}] 

pg77 

= inf sup Et [Xpl{p<p} + FM{p>0}] , 

'^GTt pg77 


9 := inf{t > p : 14 > Ft — e}. 


(3.7) 


(3.8) 


Then {p,9) £ {Tp)‘^ is an e-saddle point for the Dynkin game Vp defined in (13.7|) with t replaced by 
p. By Lemma [331 there exists h^ > 0 such that for any t > 0, there exists (ft = {fl= 

((ft being an e-optimizer for Xp for any t' £ [t — h^,t]. Similarly, by Lemma [3Tl there 

exists hy > 0, such that for t > 0, there exist (pt = (/5?,P?,3)>? = (Pt = (phPtp)^'^^ = 

£ (Tt)^ being e-saddle points for Fp and Fp respectively for any t' £ [t — L^,t]. Further¬ 

more, by Lemma 13.41 there exists h^ > 0 such that for any t > 0, there exist Pt^,ft^,at‘^ £ Tt being 
e-optimizers for 


Zf := sup Ep [U{p,t',t')] 

P^Tp 


<713 


= inf Ep [U{t',T,t')] , Zl? := inf Ep [U{t',t',( t)] 
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respectively for any f € [t — h^,t]. Let h := A A , and 

^ := K > yi - 4 . 

For any t > 0, define 

Mt) ■= {[t/h] + l)h 
Observe that (/>*(•) is right continuous and 4>h{t) > t ior t < 00 . 

Recall p defined in (13.81) . Now define p = {p, p 2 , pz, = (f, n, fa, ri 3 ),(T = (d, di, (f 2 , (fi 2 ) as 

follows. 

0, on A, 


(3.9) 


r := 


00 , on A'^, 


and d := 


and 


h{t) ■= h{t) ■■= h3{s,t) := < 


and 


■= ^3(i) := n3(s,t) := < 


<Ph{t)^ 


and 


:= dj^(^), d2(t) := di2(s,t) := < 


4>h{t)' 


6, on A^, 


00 , on A, 


/ 

4(^),3W’ 

if s < t, 


if s > t, 

cP 

if s = t, 


if s < t, 


if s > t, 


if s = t, 

/ 

if s < t, 


if s > t, 


if s = t. 


It can be shown that p, f, d G T^. In the rest of the proof, we will show that (p, (f, d)) is a 17e-saddle 
point for (|3.5p with t replaced by p. 

Part 1. We show that 


[Uip[T,a],f[p,&],a\p,T])] -Vf,\ < 8e, 

where V is defined in (13.71) . We consider five cases. 

Case 1.1: Ai := {p < 9}. 


(3.10) 


E/.[C7(p[f,d],f[p,d],d[p,f])lAi] = ^A¥.p\u(p,f\ 


<Ph{p) 


^<t>h{p) 




^4>h{p) 




G [lE^ [X^l^J + e] . 

Case 1.2: A 2 := {p = 0} H A. It can be shown that for any t >0, 


Hence, on A, 


Xt < Zl‘^ < Y^. 


Zf<Yl<V^ + e<Y^ + e. 


Then 

[[/(p[f,d],f[p,d],d[^f])lA2] =E^ 


E^ 


u[9,e,a 


12 

MS) 


f Ao 


(3.11) 

(3.12) 

G [E^ [X^Iaj] ,E^ [y^lAa] + 2e] . 
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Case 1.3: := {/5 = 0} H It can be shown that for any t > 0, 

Xt < < y/. 

By the dehnition of 6 in (I3.8p . we have that on A^, 


(3.13) 


This implies that on A'^, 


Then 


Y^^AY^^-e<V^<Y^^-e. 


Yi>Yi = YiAYi = Y^. 


[U{p[f,a],f\p,a],a[p,f])lA3] = 

Case 1.4: A 4 := {/5 > 0} H A. By (|3.12p . we have that 


1j43 


(3.14) 

G [E^ [Xgl^g] ,E^ [l^elAa] + e] . 


^ii[U{p[T,&],T[p,&],&[p,f])lA^] = E^ 


E;; 


U P] 


E,, 




M9) 

- e,IEp 


M8) 


,9,a 


YilA, 


Me) 

+ e 


Me) 


1^4 


Ep 


[ Me) \ 

- 

'Me) 

.^Me)_ 


1A5 


C [E^ [ 1 ^ 91 ^ 4 ] - e,E^ [ 1 ^ 01 ^ 4 ] + 2e] 
Case 1.5: A^ := {/5 > 0} n A^. By (I3.14p . we have that 
Ef,[U{p[T,a],f[p,a],a\p,T])lAs] = E^ 

e KK"^A,]-e,E^[Y?lA,]+e] 

C [E^ [^'^ 1 ^ 4 ] “ e, E^ [TplAs] + e]. 

From cases 1.1-1.5, we have that 

' - 2e 

< E^[u{p[T,a],f\p,a],a\p,f])] 

T ^ Yfj, + 8e. 


Vf,-3e < Ef, 


< E^ 


^p^{p<e} X ^e^{p>e} 


Part 2: We show that for any p G Tf, 

E/x [u{p[T,a],f[p,a],a\p,f])] < v^ + Qe. 

Take p = {p, P2, Ps, P23) € T^. We consider hve cases. 

Case 2.1: Bi := {p < 9}. 

Ep [17(p['r,^],r[p,a],a[p,r])lB,] = E^ fep \u (p,tJ^(p) 


(3.15) 


’^<l>hip) 


<t>h{p) 




< Ep [Xpl^^] + e. 

Case 2.2: B 2 := {p = 9} A. By (l3Tll and (imli . 


Ep [[/(p[f,CT],T[p,a],CT[p,r])ls 2 ] = Ep 


Ea 


u{9,9,a 


12 

Me) 


Case 2.3: B 3 := {p = 0} D A‘^. By (|3.13l) and (I3.14p . 


Ep [t/(p[f,a],f[p,ff],a[p,f])ls3] = Ep 


Ea 


U 9,f 


13 


Phie)' 


1 A 2 


1b3 


— Ep [ 1 ^ 0152 ] + 2 e. 
< Ep [y^lsa] + e. 
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Case 2.4: B 4 := {p > 0\ r\ A. Define p? := (p 2 (^), P 23 (^, •)) G T?. We have that 

u 0 


Ep [t/(p['r,o-],f[p,a],CT[p,f])lBj = Ep 

Eg 

U\ 

[ol 

-2 

<t>h{e)_ 



)] 

< Ep 


+ e 





— \Xe^B4, + 2e. 



Case 2.5: := {p > 0} n A^. Define p\ 

u 

E^[[/(p[f,a],f[p,a],a[p,f])ls5] = 


:= (P 3 ( 0 ),P 23 (', 0 )) G T|. We have that 


Eg 

u\ 

\Pe 

L 4>h{e)\ 

- 

^ Me) 

[4 

■^)] 


+ e 








From cases 2.1-2.5, we have that 


< E^ 




+ 7e 
+ 8e 


< D^ + 9e. 


Part 3: We show that for any (r,£T) € 

Ep [f7(p[T,a],r[p,£T],a[^r])] >V^- be. 

Take (r = (r,Ti,T 3 ,Ti 3 ),(T = (fx,fii,<72,(T 12 )) G We consider four cases. 

Case 3.1: Ci := {p < r A fx}. 

Ep [D(p[r,rx],T[^rx],rx[p,T])lci] = E^ [E^ [D(p,r[p,cx],£x[p,T])] IcJ > E^ [^plcj 
Case 3.2: (72 := {p > r A cx} n {r = a}. It can be shown that for any t > 0, 


Then we have that 

Ep [C7(p[T,<T],r[p,<T],a[p,T])lc2] = Ep 


Et 


U 


23 


hB) 


r, T 


ICa 


> Ep[Zflc2]-6 

> Ep [y,-AalC2] - e. 

Case 3.3: (73 := {p > r A u} PI {r < a}. Define rx^ := (cx 2 (r), cxi 2 (-,r) G T^. Then 


Et 


^ {pIkB) [^r] 


P<l>hir) 


Ep[t7(p['r,£x],r[p,cx],£x[p,T])lc3] = Ep 

> W.^[Y^lc,]-e 

> Ep [yrAalCa] - e. 

Case 3.4: (74 := {p > r A cx} n {r > cx}. Similar to case 3.3, we can show that 


1^3 


(3.16) 


Ep [U{p[T,a-],T\p,a-],a\p,T])lc^] < Ep [WAalc 4 ] - e. 
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From cases 3.1-3.4, we have that 


E^[C/(p[t,o-],t[p,o-],o-[p,t])] > E^[Xpl 

{P<tA(t} ^^Acrl{p>rA(T}] 


> 

> 


E^ 

y. 


^p^{p<e} 

- 5e. 


+ h^Ao-l 


{p>6»} 


-4e 


By (l3.10F(|3.13p and ()3.16p . {p,{f,a')) € is a 17e-saddle point for (j3.5p with t replaced by 

p. □ 


4. Proof of Theorem 12.61 

In this section, we will construct an e-Nash equilibrium of the three-player game by using e-Nash 
equilibriums of two-player games, as well as e-saddle points of games like (|3.5p . 

Let e > 0 be fixed. By lemmas 13.2113. 41 we can choose h > 0, such that for any t, there exist 
(tj = (r|,r^ 3 ),ffj = (u|,ctJ 2 )) € (Ti)^ being an e-Nash equilibrium for the game 

Ep [W{t',T[(T],cr[T])] , T,aeTt', i = 2,3, 

for any t' € [t — h,t], and (p^ = {'pf ,'p^ G (Ti)^ being an e-Nash equilibrium for 

the game 

Ep [U\p[(j],t',cr[p])] , p,creTt>, i = l,3, 

for any t' € [t — h,t], and (p^ = {PuPt 2 )^^t — i)) ^ being an e-Nash equilibrium for 

the game 

Ep [W{p[T],T[p],t')] , p,TGTt/, i = l,2, 
for any t' G [t — h,t], and G Tt being an e-optimizer for 

M^Ep [W{p,t',t')] 

for any F G [t — h, t] for f = 2, 3, and G Tt being an e-optimizer for 

inf Ep [U\t',TT')\ 

tSTj/ 

for any F G [t — h, t] for i = 1, 3, and G % being an e-optimizers for 


inf Ep [U\t',t',a)] . 

cr&Tfi 
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Recall 4>h{-) defined in (13.9p . For i > 0, define 

Xl := inf Ef \U^{t,T[(T],a\i 


T,a-&Tt 


:= E* 




fphit) 


’<Ph{t) 


’%Ri) 




Y^1i2 

-'t 

:= Ef 






)] 

■'t 

:= Ef 



)r3 


Pi>h{t) 

,t)_ 


:= y/’^AF/’^e 

1 







F/ := sup inf Ef [^pl{p<A} + 1{p>a}] , 

p£Te AG/e 

:= inf{f >9: F/ < + e}, 


and y/’\ y/’^, yf2, Ff2, Z^, y/’\ y/’^ y/, F^^, //^ in a symmetric way. 

Lemma 4.1. For i = 1,2,3 and any t > 0, XI < . 

Proof. For f > 0, let p = f = {t, oo) e . Then 


xl < Ef U^(t,f 




= Ef 
= Ef 


I'T 


C/Mp 




i) 

[pi) 


<y/’Ve. 


Similarly, we can show that Xf < + e. Hence, Xf < Yf. 

Lemma 4.2. For i = 1,2,3, the processes [Zl)t>o and {Yf)t>Q are right continuous. 


□ 


Proof. Let t G [0,oo). Let \ t. Without loss of generality, we assume that \tn — < (1/^^) A 

ifhit) - t). Then 


\zl -z}\ 

I on t \ 


< 


Ef 


Ef„ 




<Ph{t) 


’fphit) 






{t,T\ 




1 

—1 

1 

^<Ph{t) 

’^4>h{t) 



Since 


E 


'4+s 


t/1 (t,T^ 


4>h{t) 


-Ef 

-Ef 






(phit) 


4>h{t) 




4>hit) 


(, —1 

1 

—1 

—1 






+ r]{l/n). 






s>0 


is a martingale w.r.t. the filtration {J't+s)s>o satisfying the usual conditions, it is right continuous. 

1 .^1 ..t,_ 


Hence, —)• Zf. Similarly, we can show that (l^^’"^)f>o and (y/’'^)f>o are right continuous 


□ 


By Lemmas 13.61 and 14.21 we have that for i = 1,2,3, the process {Vf)t>o is right continuous. 
Then we can choose 5 > 0 being J^p-measurable, such that for i = 1, 2, 3, 


Eg 


sup 

0<r<(5 


_ yi 


< e and Eg 


y*. — y*. 

'^p»+(5 


< e. 


(4.1) 


It can be shown that for any A € 7p, A + <5 G Tg+. 
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By Proposition EZl there exist (p^ (r^ = (r\rj, r^, = {a},q\, 0 X 2 ))) ^ 

being an e-saddle point for the game 


P/, 1+5 := sup inf E 1+5 [C/^(p[r,a],T[p,a],a[p,r])] 


(4.2) 


and (T 2 ,(p 2 = = (£^£?,£i,£? 2 ))) ^ O^l^+s)^ being an e-saddle point for the 

game 


sup 


inf E^ 2+5 [U^{p[T,a],T\p,(T],(Tlp,T])] 


and {cr^,{p^ = (P^> P^jPa)£ 23 ^’-^ ^ (z^^ZiiZliZis))) £ (Tj^3+5)^ being an e-saddle point for the 
game 

sup inf E3^s[U^{p[T,a],T[p,a],alp,T])]. 

Now define p = (p, p 2 , P 3 , p 23 ),f = (f,fi,f 3 ,fi 3 ),d- = (d, di, ( 72 , (T 12 ) as follows. 


p := 


p^, on A, 
p^, on B, 
p^, on C, 


P2(t) ;= < 


p2(t), on B n{t > + 6}, 


on Cn {t > p'*-h(5}, 






otherwise, 


hit) := < 


and 


Mt) ■= < 


plit), 

on B n {t > p^ -|- 5}, 


PgCi), 

on C n {t > p^ -|- (5}, 

P23(s,t) 

flhity 

otherwise. 



✓ 

p^, on B, 


f := < 

r^, on j4. 

nit) ■= 


r^, on C, 

K 


ilit), 

on A n {t > p^ -|- (5}, 


zlit), 

on C n {t > p^ -|- (5}, 

Tl3(s,t) 

r3 

1 >h(i)’ 

otherwise. 



P23('S,i), 

on B n {p^ -|- 5 < s A t}. 

pl^isA), 

on C n {p^ -|- (5 < s A t}. 

3,23 

£-4‘h{t)^ 

on B n {s = t = p^}, 

2,23 

on C n {s = t = p^}, 

'pIas),S)^ 

on L) n {s < t}. 


on ZD n {s > t}, 


T}(t), on A n {t > p^ -|- 5}, 
rf(t), on C n {t > p^-|-5}, 


r <i>hW’ 


otherwise. 


Zi 3 ('S,i), on A n {p^-|-(5 < s A t}, 

7 ^ 3 ( 5 , t), on C n {p^-|-(5 < s A t}, 

on A n {s = t = p^}, 
on C n {s = t = p^}, 
^ 0 ;,(s), 3 W’ on.En{s<t}, 
on.En{s>t}, 


t-3,13 

—<f>h{t) 

1,13 
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and 


/ / 




on C, 



on A n {t > -g 

(7 := < 


on A, 

II 

<b 

dit), 

on B n {t > p'^ + (5}, 



on B, 



otherwise. 


(72 (t) := < 


gilit), on ^4 n {t > + (5}, 

g^{t), on B n {t > g? + 6}, 0 - 12 ( 8 , 1 ) 


= 


^Mt)’ otherwise, 


on ^ n + 6 < s At}, 

on B n + 6 < s At}, 

on A n {s = t = fj,^}, 
on B f] {s = t = g?}, 
on Fn{s<t}, 
onFn{s>t}, 


Oi2{s,t), 

Zi2(s,t), 

2.12 

— 

1.12 

— 


where 


A := {fj,^ < /i^ / < g^}, B := {fi^ < < g^}, C := {/i^ < /i^ < /i^}, 


and D (resp. E,F) is the complement of the first four cases in the definition of P 23 (resp. 713 ,( 712 ). 
It can be shown that p,f, & € T^. The next result shows that (p,f,a) is a 13e-Nash equilibrium for 
the game (I2.1|) when N = 3. In particular, Theorem 12.61 holds for N = 3. 


Proposition 4.3. {p,f,a) G (T3)3 is a 13e-Nash equilibrium for the game (12. ip for N = 3. 


Proof First, we have that 

Kg[u\p[f,a],f[p,a],a[p,f])] = Eg 

+ Eg 
+ Ee 
= Eg 




E^2 


E^3 




i>h{A) 


vhiA) 




4>h(A) 


A ,2 






pIh(u^) 


''^4>h{u^) P4>h{tP) 




ziu + ryiB+Yyic 


u 


Ia 

Ic 


Now take p = (p, P 2 , Ps, P 23 ) G Tg. We will show that 


Ee [u^{p[f,a],f[p,a],a[p,T])] <£0 [u^(p[f,a],f[p,a],a[p,f])] +i3e. 


(4.3) 


We consider seven cases. 

Case 1: Di := {p < A A p^}. As Xf < Zf for any t > 0, 

p^ < inf{t > 6 : < X} + e}. 
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Therefore, the process {Vt)t>o is a sub-martingale from 6 to /i^. Hence, 


Ee [u^{p[f,a],f\p,a],a\p,f])iD^] = e^ 


E„ 


(p,t1 


<t>h{p) 


^4>h{p) 


^^Mp) 


^4>h{p) 


= Ee [-^plDi] 

< ^e[V^lD;_ 


= Eg 

< Eg 
= Eg 

< Eg 






1di 


-h v\lc) 1 di 


Zi.U + Y'ihB + Ylflc) Id. 


+ 3e. 


^Di 


Case 2: D 2 := {/i^ A /\ < p < A A + 5} Ci A. By [U Lemma 2.15, page 8 ], 

{/i^ A p'^ A < p} n A € Tp, and thus D 2 € Tp. Then by (|4.1D . 


Ee[U^{p[f,a],f\p,a],a\p,f])lD2] = E^ E^ 

= Ep [2"^ 1 ^ 2 ] 


Vh{p) 


'(phip) 


^4>h{p) 


1d2 


< Eg 
= Ep 


■^ul ^Dn 


+ e 


Zl,U + Y'/lB + YlAc]'iD, 


+ e. 


Case 3: := {p > p^ A p^ A p^ + 5} r\ A. Recall V^i +5 dehned in (I4.2p . By Proposition 13.71 


Hpi +5 = sup inf Ep+s 
per^i+.AGT^i+i 


^P^{P<A} + ^A^l{p>A} 


where 


y/ := sup inf Et [U^{p[a],t,(T[p])] A sup inf E* [t/^(p[r],r[p],t)] 

VpeT2 / \ pgT? rST? 


We have that 


sup inf Et [U^{p[(T],t,(T\p])] < supEt 

pgT? ‘^eT? 






W) 


<y/’Ve, 


and similarly suppg-j -2 inf.,.g ^2 E* [[/^(p[r],T[p], f)] < -)-e. Therefore, for any t > 0, T/ < T/, 

and thus 




< H 


pi+S ^ VW5- 
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Then by (I4.ip . 

Ee [U^{p[f,a],f\p,&],a[p,f])lD 3 ] = E 0 [E^i +5 [U^ (p [T\q}] [p,q}] [p,r^])] ID 3 ] 

^^ 1 + 51^3 
V^^\+51d3 


< Eg 

< Ee 

< Eg 

< Ee 
= Eg 


V^D, 

1^3 


+ e 

+ e 

+ 2 e 

+ 3e 
. 1,2 


Z\lA + YyiB + Yyic)lD, 




+ 3e. 


Case 4: 1)4 := {p = A A p'^} fl B. It can be shown that 
> mf Et [U\t,t,a)] - e > E* [u^ 


- 2 e. 


Therefore, 

Kg[u\p[f,a],f\p,a],a[p,f])lDi] = Ee 

< Eg 

= Ee 


E^2 


rrl / 2 2 1,12 

U ip ,p ,£^^(^ 2 ) 


lil4 


b;^2'iii4 


+ 2e 

Z^U + T?ls + T^^^lc) 1 d 4 


/4‘ ^ ■ /4' 

Case 5: £>5 := {p > p^ A p^ A p^} Pi B. Let p 2 := (p 2 (p^), P 23 (p^, •)) € 


+ 2 e. 


E0[u\p[f,a],f[p,d],aip,f])lD5] = Eg E^2 C/Mp^2 






Id. 




+ e 


< E6I 
= E^i 

Case 6: Dq := {p = p^ A p^ A p^} fl C. Similar to case 4, we can show that 


Z\lA + Y^flB + Yyic) Id. 


+ e. 


Ee [U^{p[f,a],f\p,a],a\p,f])lDe] < Eg + TTls + Icj 

Case 7: Dj := {p > p^ A p^ A p^} fl C. Similar to case 5, we can show that 


+ 2 e. 


Eg [U\p[f,a],f\p,a],a[p,f])lDr] <£0 |^(^Z^i1a + T 2 Icj IdtJ +e- 

From cases 1-7, we have (14.31) holds. Similarly, we can show that for any T,cr £ T^, 

Eg [u‘^{p[T,a],T[p,a],a[p,T])] <E 0 [u‘^{p[f,a],f[p,a],a[p,f])] -bi 3 e, 

and 

Eg [t/^(p[f,£T],f[p,£T],£T[p,T])] < Eg [t/^ (p[f, (t] , f [p, (T] , (T [p, f ])] -b 13e. 


□ 
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